Plato and the Nerd: The Creative Partnership of Humans and Technology by Edward Ashford Lee

Author:Edward Ashford Lee [Lee, Edward Ashford]
Language: eng
Format: epub, pdf
Tags: Technology & Engineering, Engineering (General), Inventions, social aspects
ISBN: 9780262341219
Google: GYc3DwAAQBAJ
Publisher: MIT Press
Published: 2017-09-29T00:32:33.017013+00:00

H(X) is the information we would gain with a perfect observation, and H(X|Y) is the information that is not revealed by the experiment. In other words, H(X|Y) is the remaining randomness after the measurement. The truly astonishing thing about this theorem is that for a wide range of models of measurement noise, the difference H(X) − H(X|Y) represents information that can be encoded with a finite number of bits, even if the original outcome x of experiment X cannot be encoded with a finite number of bits. The information revealed by the experiment, in bits, is finite, although the information in the actual system, in bits, is infinite. In forming the difference H(X) − H(X|Y), both quantities have an infinite offset compared with discrete entropy, but the offsets cancel, and the difference becomes a discrete entropy. This insight is truly remarkable.

For our particular example, where a = 4, we have determined that H(X) = 3. Calculating H(X|Y) precisely is a bit tedious, so I will spare you the details, but our intuition holds up, and H(X|Y) turns out to be slightly less than 0. Hence, C in equation (4) turns out to be slightly larger than 3, indicating that our measurement reveals slightly more than 3 bits of information on average.

It is now worth considering some special cases. Suppose the measurement is perfect. In this case, H(X|Y) is minus infinity because once a measurement is taken, there is no remaining randomness in X. Hence, C is infinite regardless of the value of H(X) (as long as H(X) is not also minus infinity). As a consequence, a perfect observation of a continuous random experiment yields an infinite number of bits of information.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.